DOCOHBIY IISOII 



ID 097 071 

lUTHOB 
TITIB 

ZW8TIT0TI0H 
SMfS AOmCT 
FOB BATB 
lOTB 

BDBS PBXCB 
DBSCBXPTOBS 



JC 740 394 

Horrify Williii; Gold, B«n R. 
Student Aeeottatability lod«Xs Proc«darM Hanaal. 
foeatlonal Bdaeation, Part C. 
California Coaaanity Collages, Saeraaanto. 
Offiea of Bdaeation (DREW) » laahington, O.C. 
lag 74 

56p.; Baaaareh Projaet 19-34741-C3-235 

IIF*$0.75 HC*I3.15 PL08 P08TA6B 
collaga Stadanta; ^Coaauaity Collagas; Dropoat 
Baaaaroks B?alaatioa Rathods; *Pollovap Stadiaa; 
Gradaata Sarvaya; Qoidaa; *Hodala; Poat Sacondary 
Education; *Prograa Bvalaation; Qaastioniiairaa; 
Saapling; Stadant charaetariatics; «Yoeational 
Bdaeation 

IDBBTZFZBBS Calif or niai SIR; «StQdant Aeeoantability Hodal 
ABSmCT 

Tha Stadaat Accoantability Hodal (SIR) vaa datalopad 
by a coaaortiaa of 12 aaabara, to provida a aystaa of procadaraa for 
idantifying and daacribing California coaaanity collaga occapational 
atadanta and for obtaining inforaation about thaa aftar they laata 
collaga. Tha two coaponanta of tha aodal ara tha stadant Accounting 
Coaponant (Claaaification of Occapational Coaraaa, Zdantification of 
occapational Studant Rajora, and Data for foc*Bd Por CC0B48) and tha 
Stadant Followap Coaponant (Claaaifidation of Pratioua Saaaatar 
stadanta, Follovap Procadaraa for Oroapa— aalaction of groap(a), 
praparation of atadanta, atudy daaign**Data for foe*Bd Pora CC0B45t 
and Data for Prograa BYaluation and Planning). Bach of tha taaka 
involvad in tha tvo coaponanta ia diaeuaaad. (Appandiiaa praaant tha 
following: a diacuaaion of Saapling in Follovap stadiaa, Saapla 
Inatruaanta for Follovap Praparation (Xn*Claaa Qaaationnaira, BUT 
Intarviav Fora, and Follovap Hart Lattar), and a liat of aaabara of 
tha Projaet Conaortiua. (DB) 



ERIC 



UJ 



us ^EPARTMtNTOFHtALTH. 
GDUCAriONA*VtlFARE 
NATrONAL INSnri'Tt Oh 
CUUCATtON 

TH- , Hue. Vf Ni HA^ fcu t\ U€PWO 

'^'t-'' ^«0M 

'FNTO. f f A. NATION... . Ni, ! . T U T f 0» 



STUDENT ACCOUNTABILITY MODEL 



BEST COPY AVAIUBLE 



PROCEDURES MANJAL 

Vocational Education, Part C 
Project #19-34741-C3-235 



Chancellor's Office 
California Cotnminlty Colleges 
William Morris/ Evaluation Specialist 



Ben K» Gold, Director of Research 
Los Angeles City College 
Project Coordinator 



Auguit, 1974 



FRir 



Tae activity which Is the subject of this report 
vas supported In whole or in part by the U* S. Office 
of Education 9 Department of Healthy Education » and Wei- 
fare* However » the opinions expressed herein do not 
necessarily reflect the position or policy of the U« 
Office of Education 9 and no official endorsement by the 
U« Office of Education should be Inferred* 



TABLE OF CONTENTS 

Page 

Preface 

Steps In Implementing Model 

Introduction •••••••••• •••••••••••••••••••••• 1 

I Student Accounting Component •••••• 4 

A. Classification of Occupational Courses • 4 

B. Identification of Occupational Student Majors** • • • 8 

C. Data for Voc-Ed Form CC0E48 (Enrollments) 11 

II Student Follow-up Component 15 

A. Classification of Previous Semester Students 15 

B. Follow-up Procedures for Groups 18 

1. Selection of Group(s) • 18 

2. Preparation of Students 19 

3. Study Design 20 

C. Data for Voc«Ed Form CCOE45 (Follow-up) 26 

D. Data for Program Evaluation and Planning • 31 

Epilogue 37 

Appendices 

ERLC 



4 



PREFACE 



This manual* Is a product of 1973-74 Vocational Education 
Part C, Research Project #19-64741-03-235, ••Improving Occupa- 
tional Student Follow-up In California Connunlty Colleges The 
major thrust of that endeavor was the construction of a model to 
accomplish the purpose stated In the project title* 

It became evident early In the project that a fundamental 
component In the model would have to be a uniform method for 
classifying occupational courses and identifying occupational 
majors so that non- continuing students could be categorized for 
various approaches to follow-up. This manual describes recom- 
mended procedures for accomplishing these objectives and suggests 
methods for obtaining useful Information about occupational stu- 
dents during and after their stay in college* 

The model was developed by a consortltim of twelve leaders 
in California Community Colleges, including occupational and 
general administrators, researchers, and data processing special- 
ists* Input was solicited from other colleges and from staff of 
the Chancellor's Office. 

It is hoped that the model will be of assistance to the col- 
leges in (1) evaluating and planning their occupational programs 
and in (2) completing required enrollment and follow-up forms re- 
lated to vocational education* 
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^ A separate manual "Flowcharts and Programs for S«A«M/^ is 
aval lab le« 
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STEPS IN IMPLH4ENTING OCCJPATIONAL STUDEtTT ACCOUNTABILITY MODEL MODEL (SAM) 



TASK 

I. Accounting Component 
1« Classify Courses 

Identify all occupational courses 
Label each with appropriate CID 
or USOE (VEA) number 
Label each with priority letter 

2« Identify student occupational majors 
Examine student ^s program 
Place student In Class lf2,3, or 4 
Identify major by CID or USOE code 
for each student In Class I $2 ^3 

3« Prepare student Occupational Majors 
List (OML) 

4« Prepare annotated OML (or ex- 
panded SM^) 

Merge OML with Student Master 
File (SMF) to Identify de- 
sired student characteristics 
with major and class 

5. Make counts for CCOE48^ 

Make Fall counts and store; make 
Spring counts; get total ur.du- 
plicated counts by totaling 
Fall and Spring counts, eliminat- 
ing Fall figures for students 
enrolled both semesters 

II. Follow-up Component 

6« Identify non-continuing students 
Match current SMF against pre- 
vious semester SMF (as of census 
week) 

^Alternative method for step (5) 
Make Spring counts; add Fall 
counts for non- continuing stu- 
dents only (requires Step (6)) 



TIME 



Prior to each 
semester 



PROCEDURES 

MANUAL 
REFERENCES 



I-A p. 4-7, 

10 



Census week, each I-B p. 8^10 
semester 



Census week, each I-C p« 11 
semester 



Shortly after I-C p# 11-14 

census week, each 

semester 



Semester counts I-C p. 11-14 
shortly after cen- 
sus week, each 
semester. Total 
counts after Spring 
census week 



Census week, each II-A p« 15 
semester 
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TASK 



7. Prepare Follow-up Matrix 

Horizontal position (Class 1,2,3, 
4) determined by step 2 above 

Vertical position (G,C,A,P,N) 
determined by accomplishment 
during previous semester 

Each non-conwinulng student 

placed In appropriate cell of 

matrix 



TIME 



PROCEDURES 

MANUAL 
REFERENCES 



Shortly after II-A p. 15-17 

census week, each II -B p. 21 
semester 



8« Select cells of matrix for 
follow-up 

Select according to college 
needs; must Include groups 
for CCOE45 

9« Perform follow-up procedures 
Design study, collect and 
analyse data, disseminate 
findings 



10. Cc^iuplete CCOE45 

Total counts for Fall 
and Spring 



Shortly after 
census week, each 
semester 



Shortly after 
census week. Fall 
semester (also 
Spring semester If 
desired) 

November, each 
year 



II -B 


p. 


18-21 


II-D 


p. 


31 


II-B 


p. 


21-26 


II-C 


p. 


26-31 


II-D 


p. 


31-36 


II-C 


p. 


26-30 



********** 
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STUDENT ACCX)UNTABILITY MODEL 



INTRODUCTION 

The model described herein was developed to accomplish the following 
general purposes: 

To provide a system of procedures for Identifying and describe- 
Ing California community college occupational students and for 
obtaining Information about' them after they leave college, In 
order to satisfy two needs: 

(1) completion of reporting forms re* 
quired for governmental agencies, 
and 

(2) program evaluation and planning 

Visits to a statewide sampling of colleges during the early planning 
stages revealed a wide variety of practices in obtaining quantitative 
information regarding occupational program enrollments and subsequent 
Job placements^ These visits also produced data, and opinions, con- 
cerning needs and operating constraints within individual colleges* 
As a result of these visits, the following sets of guideline^; and as* 
sumptions were developed as bases for construction of the model* 

Guidelines for the System: Essential 

(1) System must be adaptable to any California community 
college and to all occupational programs offered by 
the college. 

(2) Operation of system must be sufficiently simple that 
outside expertise is not required after initial imple- 
mentatlon« 

(3) System must lend Itself to electronic data processing 
but be adaptable to manual operations* 

(4) System must maximize the use of current college staff 
and minimize the necessity for hiring addltiot^al pei^son* 
nel* 

(5) System must be ongoing, not "one*shot*" 

(6) Any alternative sub-systems must provide substantially 
equivalent information* 

(7) System must provide necessary information for Vocational 
Education enrollment and follow-up forms with a high level 
of accuracy and on time* 
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Guidelines for the System; Essential (continued) 

(8) System must provide pertinent evaluation and planning 
Information to occupational administrators and faculty. 

(9) System must Indicate enrollments as of the census week 
In the semester (quarter) concerned, but must also pro- 
vide flexibility for specially scheduled courses # 

(10) Data collection instruments must be easily readable, re- 
quire a minimum of time for completion and be in a format 
easily adaptable to electronic data processing. 

r 

Guidelines for the System: Highly Desirable 
System should: 

(1) be expandable to Include students other than occupational 
students. 

(2) provide for employer as veil as employee input. 

(3) provide for long-term as well as short-term follow-up. 

(4) require a minimum of effort for colleges to "retool" from 
existing systems. 

(5) not in any way imply that other methods of program evalua- 
tion are less important. 

(6) build on existing research information. 

(7) adapt with minimum difficulty to changes in reporting re- 
quirements . 

(8) provide estimates of any sampling and/or response bias. 

(9) provide additional Information useful to college counsel- 
ing staff. 

(10) provide additional information useful to college placement 
office. 

Some Assumptions and Inferences 

(1) Colleges need a system such as proposed in this project. 

Inference: Colleges will be committed to participa- 
tion and support of the project. 

(2) No system for identifying student majors can be 1007. ac- 
curate. 

. Inference: Project should determine what accuracy 
is reasonable and model should assure 
that accuracy. 
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Some Assumptions and Inferences (continued) 

(3) Difficulty In student major Identification decreases the 
longer the studert stays In college. 

Inference: Target accuracy rates should vary with 
groups Involved. 

(4) No sizable follow-up procedures can produce 100% response. 

Inference: Project should determine what response 

rates are acceptable and under what cir- 
cumstances. 

(5) Difficulty In contacting former students Increases the 
longer they are away from college. 

Inference: Target response rates should be expected 
to vary with groups Involved. 

(6) A workable system requires Input from those ''close to the 
scene.'' 

Inference: Occupational faculty, department or division 
chairmen, and counselors should be Involved 
In the system* 

(7) Follow-up Information aboitt students who have completed 
considerable work at the college lo more useful In pro- 
gram evaluating and planning than that about students who 
have completed little or no work. 

Inference: While short-tenure students should certainly 

not be Ignored, more effort should be expended 
on the follow-up of long-tenure students. 

(8) Specific objectives of follow-up studies may differ among 
colleges and among groups within a college. 

Inference: Specific objectives for follow-up studies 
should be determined before the study de- 
sign Is finalized; over-all system must be 
flexible enough to adapt to differing ob- 
jectives. 

(9) Student attitude Is an Important factor In follow-up re- 
sponse rates. 

Inference: Efforts should be made to prepare students 
for follow-up requests, preferably before 
they leave the college. 

(10) A large follow-up response from a representative sample Is 
preferable to a small biased response from an entire group. 
Inference: Repeated efforts to obtain responses should 
be made. 
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To meet these guidelines and asstsnptlons^ the model was envisioned 
as having two components: (1) the Student Accounting Component » in 
which occupational students are operationally defined and classified 
according to criteria which are both adequate and simple to apply » 
and (2) the Student Follow-up Component » In which occupational stu- 
dents no longer In college are categorized and procedures recommended 
for obtaining Information about them^ with details of procedure de- 
pending on the category^ 

I • Student Accounting Component 

In the Student Accounting Component ^ occupational courses are 
classified and prioritized, and student majors are defined ac- 
cording to the occupational courses In which they enroll* 

The most critical aspect, therefore, of the entire model Is the 
careful classification of occupational courses* 

I-A. Classification of Occupational Courses 

An occupational course Is defined as one which Is (all three)* 

(1) Intended to develop skills and related know- 
ledge needed for Job performance 

(2) part of the course sequence of an occupational 
program offered by the college 

(3) designed primarily for Job preparation and/or 
Job upgrading or updating and not for general 
education purposes 

• 

Prior to each semester, the occupational administrator (or his de- 
legate) should examine all courses offered by occupational depart- 
ments, number each with an appropriate CID (or USOE) number*, and 
follow (or precede) the number with a priority letter, as follows: 

Priority "A^* - Apprenticeship 

Course Is an apprenticeship course 

Priority •'B" - Advanced occupational 

Course Is offered In one specific occupa- 
tional area only and clearly labels Its 
taker as a major in this area* Priority 
B courses will generally (but not neces- 
sarily) be those taken by students In the 
advanced stages of their occupational pro- 
grams* 

* Nuttibers should be assigned, according to Judgment and experience of ad- 
ministrator, to specific area most consistent with content of the course* 
CID: Classification of Instructional Disciplines, State Chancellor's Office 
USOE: U*S* Office of Educational Vocational Education Classification 
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Classification of Occupational Courses (continued) 

Priority "C" - Clearly occupational 

Course Is generally offered In several 
specific occupational majors within a 
broad area and Is of difficulty level suf-* 
flclent to detriict •Mrop-lns". Priority 
C courses will generally (but not neces- 
sarily) be taken by students In the middle 
stages of their occupational programs* 

Priority ••D" - Possibly occupational 

Course Is Introductory level occupational 
or Is primarily a eervlce (or survey) course 
for other occupational programs « Priority 
D courses may be taken by non->occupatlonal as 
well as occupational majors « 

For borderline courses (between C and D) , It 
Is suggested that the deciding factor should 
be an estimate (based on judgment and experi- 
ence) of the number of students taking the 
course whose major Is judged to be occupational* 
If the number Is estimated to be 75% or more of 
the total 9 the course should be Identified as 
Priority C, otherwise Priority D. 

Priority - Non-occupational 

Although offered by an occupational department , 
course Is designed for non-occupational majors 
who desire acquaintance with the field as part 
of their general education* 

In addition to the above, letters F and beyond may be used to Identi- 
fy special courses, e*g*, useful homemaklng, working experience* It Is 
suggested that work experience be given a ^higher occupational priority 
only if it is clearly part of an occupational currlcultm* 

Also, it should be noted that in this model there is no operational 
distinction between A and B priority courses* Separate identifica- 
tion is included for those colleges who may wish to study their ap- 
prenticeship courses separately* 
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In making decisions as to which priority letter to assign to 
a course 9 note that there are three possible factors which 
might be considered: 

(1) the degree to which the course Identifies 
Its taker as an occupational major 

(2) level of the course In an occupational 
program 

(3) percent of students taking the course who 
are judged occupational majors 

It Is recommended that the following guidelines be used, and 
In the order Indicated: 



PRIORITY LEVEL 



FACTOR 


A or B 


r 


D 


E 


(1) Ability to 
Identify 
major 


Clearly labels 
taker as a ma* 
jor In sped* 
flc occupa* 
tlonal program 


Indicates the 
taker Is a 
major In one 
of several 
programs In 
a broad oc* 
cupatlon&l 
area 


Suggests that 
taker Is pos- 
sibly an oc- 
cupational 
major 9 but 
area Is un- 
known 


Indicates 
taker to 
be a non- 
occupa- 
tional ma** 
jor 


(2) Course 
level 


Advanced oc* 
cupatlonal (B) 

Apprenticeship 

(A) 


Second level 
occupational 


Introductory 
occupational 


Introduc- 
tory Non- 
occupational 


(3) Expected 
number of 
occupa* 
tlonal 
majors In 
class 


100% 


75% or more 


Below 75% 
but above 
25% 


25% or 
below 



Page 7. 



Following are some examples (for Illustrative purposes only) of 
possible classifications of courses: 



A. Apprenticeship Courses 

Carpentry, Plumbing, Machine Shop 

B. Courses Which Indicate a Specific Major 

(Advanced occupational) 

Dental Pathology, Advanced Video 
Tape, Advanced Applied Acting, Ad- 
vanced Legal Secretarial Procedures, 
Contact Lens Laboratory, Advanced 
Radiologic Technology, Fire Hydrau- 
lics, Livestock, and Dairy Selections, 
Real Estate Finance, Cost Accounting 



C. Courses Which Indicate a Broad Area Major 
(Clearly occupational) 

AREA* COURSE 



01 Agricultural Soils 

04 Distributive Education Credits and Collections, 

Principles of Advertising, 
Air Transportation 

07 Health Occupations Clinical Techniques, Princi- 

ples of Patient Care 

09 Home Economics Food and Nutrition, Sanita- 

tion/Safety 

14 Office Occupations Shorthand, Small Business 

Management, Advanced Typing 

16 Technical Education Technical Engineering, Vacuum 

Tube Theory 

17 Trade and Industrial Fundamentals of Air Conditioning 

Basic Electricity 

D. Courses Which May Indicate Occupational Major 

(Possibly occupational) 

Technical Hithematlcs, Graphic Coomunl- 
catlons. Elementary Mechanical Principles, 
Fundamentals of Electronics, Typing (Be- 
ginning or Intermediate), Accounting (Be- 
glnnlug) . 

E. Survey of Exploratory Courses (Not occupational) 

Introduction to Home Economics, Introduc- 
tion to Business, Introduction to the Theatre. 

It should be emphasized that the above are Illustrations only. Course 
title alone Is not sufficient to determine course priority as courses 
with the same title but at different colleges may be quite different. 
Guidelines stated above should be used. 



Broad areas need not be limited to the Voc-Ed categories Illustrated here. 
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I-B Identification of Occupational Student Majors 

For the planning and evaluating purposes of this model, 
student majors are classified four ways: (see diagram)* 

Class 1 -* Occupational major, specific major 
determined « 

All students enrolled In one or 

more courses labeled A or B« 

Class 2 - Occupational major, broad area de-* 
termlned* 

All students who are not enrolled 
In any course labeled A or B, but 

are enrolled In ore or more courses 
labeled C« 

Class 3 - Occupational major, area not determined* 

All students who are not enrolled 
In any course labeled A, B, or C 
above, but who are enrolled either 
In (a) one course labeled D and a 
total of 7 units or less, or (b) 
two or more courses labeled D« 

Class 4 - Non-occupational major. 

All studf *s not classified above. 

Note that part-time, full- time or day, evening distinctions 
are not made« Procedures for Identification of student ma- 
jors should be applicable to all these groupings. 



DETERMINATION OF STUDENT MAJOR 
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Is student enrolled 
In one or more 
Priority A or B 
courses? 



Yes 



Student Is: 

Occupational 
Major, 
Class 1 



No 



Is student enrolled 
In one or more 
Priority C 
courses? 



Yes 



Occupational 
Major, 
Class 2 



No 



Is student enrolled 
In one or more 
Priority D 
courses? 



Yes 



No 



Is sttident enrolled 
In 7 units 
or 

less? 




No 

f 


Is student 
two or 
Prlorlt 
cours 


enrolled In 
more 

y D 
es? 



Yes 



Occupational 
Major, 
Class 3 



Yes 



No 



Non-Oceupatlona 1 
Major, 
Class 4 
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IDENTIFICATION OF STUDENT MAJORS 
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I-C. Data for Voc-Ed Form CCOE48 

For the evaluative and planning purposes of the model, occupa- 
tional majors In Classes 1, 2, and 3 will be considered accord- 
ing to their class designation (see Section II-D). 

For the purposes of the CC0E48 (Number of Students In Occupational 
Education) reporting form, the total of all occupational majors 
(Class 1, 2, or 3) should be reported In the area Indicated by the 
CID* or USOE number. If a student's major Is not uniquely de- 
termined because of enrollment In two or more courses of the same 
priority but with different numbers, the college or district should 
determine a method for making the assignment. It Is suggested that 
students be contacted (through a teacher or counselor) requesting 
that they Indicate a choice, or. If the numbers are not too large, 
that they be arbitrarily placed by the occupational administrator, 
or by arbitrary computer assignment. 

An undupllcated listing of occupational students (OML) and their 
majors can now be prepared (shortly after census week) for each 
semester (or quarter) . 

It Is aspimed In this model that there exists (for each semester 
or quarter) a periodically updated student master file (SMF) Indi- 
cating characteristics of students necessary for Identification of 
special groups. For completion of the Vocational Education forms 
considered In this manual, the SMF should contain the Items Indi- 
cated In the table showing minimum elements needed (p. l^r). Not 
all colleges will have all these characteristics Indicated on the 
SMF. It Is suggested that these missing Items be put on the file 
where possible. For those Items which can not be put on the SMF, 
calculations (or estimates) will have to be provided by whatever 
means are available and feasible. 

The following procedures** are required to obtain the numbers neces- 
sary for the CC0E48 form: (see diagram). 

(1) merge the Fall SMF and the Fall OML to 
obtain counts for Fall 

(2) merge the Spring SMF and the Spring OML 
to obtain counts for Spring 

(3) to obtain total (undupllcated) counts, 
add Fall and Spring counts, subtract- 
the Fall figures for students on 
both semester lists (see diagram). 

(continued on next page) 

* CCOE48 requires USOE program numbers. Colleges using CID numbers can 
adjust by using CID-USOE equivalency table provided by the Chancellor's 
Office. 

** Additional (but similar) procedures are required for colleges on the 
quarter system. 
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I-C Data for Voc-Ed Form CC0E48 (continued) 



In obtaining undupllcated counts for students enrolled In both 
Fall and Spring semesters, the following Is recomnended : 

(a) If the student's major class (1, 
2, or 3) Is the same for both 
Fall and Spring, but his (or her) 
USOE numbers are not the same, 
the Spring USOE number should be 
the one used. 

(b) If the student's major class as- 
signment for Fall differs from 
that for Spring, the USOE program 
number for the higher ranked class 
(lower numerically) should be the 
one used. 



Several examples follow: 



Student 


Class 


FALL 

USOE 
Prog. No. 


Class 


SPRING 

USOE 
Prog. No. 


USOE 
Program Code 
to be used In 
CG0E48 


A 


1 


16.0399 


4 


Non-occup. 


16.0399 


B 


4 


Non-occup . 


3 


17.1501 


17.1501 


C 


2 


17.0401 


3 


17.04 


17.0401 


D 


3 


04.01 


3 


16.01 


16.01 


E 


3 


14.01 


1 


14.0105 


14.0105 


F 


2 


04.19 


2 


01.03 


01.03 


G 


1 


07.0301 


2 


07.07 


07.0301 
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MTA FOR CC0E48 (SEMESTER SYSTBi) 



FALL 



SPRING 




SMF 




OML 



Annotated 

OML 



ontmumg 
Student! 
Fall OML 





f 


continuing 


Student! 








Counts 
(A-+fi-C) 
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MINIMUM ELEMENTS NEEDED FOR STUDENT MASTER FII^ (SMF) 
(for completion of Voc-Ed forms) 

1« Name 

2. Address (Including telephone nuniber) 

3. Social Security ntnnber (or other ID #) 
4« Blrthdate 

5« Sex 

6« Current course enrollments (updated during semester) 
7. Course grades (at end of semester) 

B. Total cumulative units completed (to Identify 1st year» 
2nd year 9 other) 

9. Status: regular, adult preparatory » adult supplement 

10 • Identifiers for the following: 

(a) disadvantaged 

(b) handicapped 

(c) minority group 

(d) cooperative work 

experience 

(e) Part H Work Study 

(f) Part D Exemplary 

11. Date Associate Degree awarded (if any) 

12. Date program certificate received (If any) 
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XI. Student Followr^up Component 

In the Student Follow-up Compommt, non-continuing students are 
classified two ways: (1) according to their major class , as de- 
scribed earlier, and (2) according to their accomplishment dur- 
ing the previous semester. Thus a two way ••follow-up matrix" is 
obtained. Procedures for follow-up are recommended, with proce- 
dural specifics dependent upon the student's location in the fol- 
low-up matrix. 

II-A Classification of Previous Semester Students 

After census week, current semester SMF and previous semester SMF 
(end of semester) should be matched. The following definitions 
pertain: 

Continuing student: student named on both lists 

Non-continuing student: student nanved on previous list but 

not on Current list 

Non-continuing students are further classified as one of the follow- 
ing, depending on accomplishment during the previous semester; 



Graduate (G) received Associate Degree 

Certificate (C) received program certificate 

Achiever (A> does not qualify as graduate or 

certificate but; 

Priority A or B completed one or more priority A 

or B courses 

Priority C completed one or more C priority 

courses (no A or B) 

Priority D completed one or more D courses 

(no A, B, or C) 

Non-occupational^.... completed seven or more units (no 

A, B, C, or D) 

Partial Completor (P)..# completed some work but does not 

qualify as graduate, certificate, 
or achiever 

Non-completor (N) does not qualify for any of the 

above 



For the purposes of Form CCOE45, (Follow-up Students Completing 
Preparatory Occupational Education Programs), Graduates, Certifi- 
cates and Priority A or B Achievers will be considered to have com- 
pleted their programs (Column 2). Priority C Achievers and Priority 
A or B Partial Completers will be considered to have left the pro- 
gram with marketable skills (Column 13) . Numbers obtained for Fall 
and Spring are simply added to obtain year totals. 

^ Non- occupational majors are not Included in the model. Inclusion here 
is for completion and Is a suggestion only. 
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CLASSIFICATION OF NON-CONTINUING STUDENTS 
(Flow Chart) 



During prevtoug semester; 



Did student 
receive Associate 
Degree? 


Yes 








No 

r 






Did student 
receive program 
certificate? 


Yes 


^ 








No 






Did student 
complete one or more 


Yes 




occupational A or B 
courses? 





I 



No 



Did student 
complete one or more 
occupational courses? 



No 



Did student 
complete one or more 
occupational D courses? 



No 



Did student 
complete 7 or more 
xinlts? 



I 



No 



Yes 



Yes 



Yes 



Graduate 



Certificate 



Achiever, Priority A or B 



Achiever, Priority C 



Achiever, Priority D 



Achiever, Non-occupational 



Did student 
complete any work? 


Yes 


► 


Partial Completor 












No 


► 


Non-Completor 



BEST copy mtUBLl 

II A-1 Classification of Groups for Follov-up ; 



Page 17* 



The following groups can now be Identified for appropriate follow-up: 



Class 1 
Specific 
Major 



Class 2 Class 3 Class 4 

Broad Area not Non-Occupational 

Area Determined Major 



Graduate G 



Certificate C 



Achiever A 



Partial P 
Conpletor 



Non- 
Coropletor N 





Considered to have completed their program 
(for CC0E45) 



Considered to have left with marketable skills 
(for CCOE45) 
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II-B , Followup Procedures for Groups Identified In II -A 

II-B-1 Selection of group(s) to b3 followed: 

Choice of groupCs) to be followed will depend on the 
purposes for which the study Is made. As stated ear- 
lier, in the model two purposes are considered: 

(1) completion of reporting forms required for 
governmental agencies, and 

(2) program evaluation and planning 

For the first of these purposes (specifically CCOE45) , 
all occupational graduates, certificates, and achievers 
Clrjs 1 must be Included (also, achievers class 2 and 
partial completors Class 1 must be cotmted). 

For the second purpose, any or all categories may be 
Included, depending on the needs or desires of the 
college at the time. 

Keeping In mind the guidelines for the model stated 
earlier, following is an example of a possible pat- 
tern for short range follow-up for a large college: 



Category 


Class 


No. to Contact 


How Often 


Graduates 


1,2,3 


All 


Annually ** 


Graduates 


4 


All 


Blannually 


Certificates 


1,2(3,4)* 


All 


Annually ** 


Achievers 


1,2 


All 


Annually ** 


Achievers 


3,4 


Sample 


Blannually 


Partial Completors 


1,2,3,4 


Sample 


Blannually 


Non*Completors 


1,2,3,4 


Sample 


Blannually 



* very likely no students will fall Into these catagorles 



It Is also possible (and probably advisable) for further 
breakdowns to be made for detailed Indepth follow-up stu- 
dies « For example, one might wish to obtain more detailed 
follow-up Informatieiv on, say, electronics graduates one 
year, allied health gradxiates the next year, atCe If It Is 
not possible to make studies on all non-contlnulng students 
routinely, such procedures could provide balanced Information 
over a period of a few years # 

^ Annual follow-up Is required by CCOE45e More detailed study could be 
scheduled less often # 
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Responsibility for follow-up will depend upon the general 
assignment of responsibilities at the college. At some col- 
leges, It may be appropriate for the Occupational Dean's Of- 
fice, at others the Researdi Office, at others the Counseling 
or Placement Office. Wh^re specific program majors are to be 
probed In depth, It Is often appropriate for faculty or depart- 
ment chairmen to conduct the study. If sampling Is to be used, 
someone with a clear understanding of bias and sampling error 
should be Involved. 

It should be emphasized that the above pattern lo presented as 
only one of many possible patterns. It should also be emphasi- 
zed that without a carefully designed pattern and commitment to 
Its Implementation, follow-up Is likely to be haphazard and 
piecemeal, with a low probability of producing solid, useful In- 
formation for program planning and modification. 

II-B-2 Preparation of Students for Follow-up 

Researchers who have performed follow-up studies consistently 
report that preparation of students while still In college pro- 
vides measurable and significant Improvement In (1) response 
rate to mailed questionnaires, and (2) completeness and accuracy 
of responses. Efforts are apparently especially needed for 
young students and for non-completors. 

One writer has characterized two types of recommended prepara- 
tion: 

(a) mechanical. Involving such Items as 
verifying and/or updating addresses, 
completing EXIT forms, and generally 
making sure that pertinent Informa- 
tion Is on file at the college while 
the student Is there to provide It, 
and 

(b) psychological, or •'conditioning" the 
student to respond to future requests 
for information. 

Following are some suggestions: 

(1) to alert Class 1 or Class 2 majors (all categories 
except N), allow 10-15 minutes of tine In each A, 
B, or C priority class shortly before the end of the 
semester and 

(a) inform students of possible future 
contacts, emphasizing 

college interest In Individual's 
opinions 

requirements for obtaining ftmds 
opportunity to assist future stu- 
dents 

no individual identification in 
any published reports 



1. 

2. 
3. 



4. 
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(b) suggest student Include college 
among people to be notified when 
he changes address — point out 
various benefits to him when 
college can contact him easily^ 

(c) If possible show student copy of 
questionnaire he may receive* 

(d) obtain Information about plans 
for next semester , current ad- 
dress and phone number , and ad^ 
dress of someone who will al- 
ways know fttudent^s address 
(see Sample In-Class Question- 
naire) « 

Note that It Is possible to obtain considerable **follow-up** lnforma«- 
tlon before the student leaves college* For example, students who 
already have Jobs lined up can provide CCOE45 Information and thus be 
excluded from later mailings for that purpose* 

(2) to alert Non-Completors, Institute a procedure requir- 
ing Exit Interviews* At the time the student withdraws 
obtain pertinent Information atid alert student to pos- 
sible follow-up requests* (see Sample Exit Interview 
form)* Although some students will Just disappear ^ ' !^ .s 
number can be minimized by consistent college-wide ef- 
forts, e*g*, annotmcements In registration packet , col- 
lege newspaper, bulletin boards, etc* urg'fng students to 
protect their records by following appropriate withdrawal 
procedures * 

(3) to alert others, send letter as soon as non-attendance 
Is determined, explaining Importance of feedback Infor- 
mation and requesting student to (a) keep college posted 
as to his address, and (b) respond when contacted (see 
Follow-up Alert letter) « 

As In other aspects of the model. Individual colleges and Individual pro- 
grams within a college may require different approaches* The above are 
listed as suggestions to be tried when deemed appropriate, but the princi- 
ple of preparation of students for follow-up while still In college has 
been demonstrated to be sound and It Is strongly recomnended that vari- 
ous procedures be tried with enthusiasm and creativity* 

-B-3 Study Design 

It Is obvious that one particular detailed plan for following 
non- continuing students cannot suffice for all students In all 
programs In all colleges at all times* 
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Study Design (continued) 

However I research and experience Indicate that the general 
procedures Indicated In the accompanying flowchart are usual* 
ly approprlr.te and feasible. Specific procedures for each 
step in the flowchart will depend upon inany things , Including 
numbers of students Involved, available resources , time frame 
limitations 9 types of Information desired, and. In general, the 
purpose of the particular study. 

Notes on General Flowchart for Followup 

(1) Prepare Follow-up Matrix 

The follow-up matrix described In Section II-A can be prepared 
shortly after census week In any semester (or quarter), as 
soon as non-contlnulng students can be Identified « The stu- 
dent *8 major class (horizontal In the matrix) should have been 
Identified during the previous semester, and his non-contlnulng 
status (vertical In the matrix) Is determined by his accomplish- 
ment that semester* Each student can now be Identified by a 
number (1, 2, 3, or 4) Indicating his major class and by a letter 
(G, C, A, P, N) Indicating his non-contlnulng status. There are 
thus twenty cells In the matrix Into which a student could be 
placed, and a simple count would produce the number of students 
falling Into each. Depending upon the collegers long-range 
follow-up plans. It may or may not be advisable to analyze In 
further detail (by specific major) the distributions In the vari- 
ous cells* 

(2) Decide on Purpose (s) of Study 

After observing the nxtxribers In the various cells of the follow- 
up matrix, specific purposes of the study should be formulated. 
These purposes may or may not be dependent upon these nxsnbers. 
Purposes may have been clearly delineated In a prepared detailed 
follow-up plan such as suggested In Section II-B-1, or they may 
be dictated by the numbers in the cells. For a particular semester, 
the purpose may be simply to obtain Information of the CCOE45 form, 
or It may be to analyze In depth graduates of one or more particu- 
lar programs, or It may be to obtain Information regarding non-com- 
pletors — or a variety of other purposes. 

(3) Select Group(8) from Matrix 

If the purpose of the study Is limited to obtaining data for com- 
pletion of the Vocational Education CCOE45 form, certain specific 
cells In the matrix will have to be chosen (see Section II-C). If 
the purpose Is program evaluation and planning, selection of groups 
will depend upon the purposes decided upon In step (2). (Some sug- 
testlons are offered In Section II-D). 
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Notes on General Flowchart for Follov^up (continued) 

(4) Assign Responsibilities 

As Indicated earlier, general responsibility for follow-up will 
depend upon situations existing at the particular college (or 
district) • At this stage In the flowchart It Is essential that 
one particular Individual (or several, If the work is to be 
partloned by program or department) be designated to direct the 
details of the particular study. It Is also Important at this 
stage for the study director to know what resources will be avail- 
able to him, In terms of budget, personnel, time and data proces- 
sing facilities. 

(5A) Prepare Students for Follow-up 

One of the first assignments of the study director should be to 
determ5rie what has been done In the way of student preparation for 
follow-up, assess Its scope, and plan to remedy as much as possible 
any deficiencies he may find. It Is to be hoped that after a 
semester or two, each college will have developed and Implemented 
plans for follow-up preparation along the lines suggested In Sec- 
tion II-B-2. The study director should be aware of what has been 
done and realize that preparation Is possible until data Is acttial- 
ly collected from the students. 

(5) Determine Sample Size 

After the decision Is made as to which cells In the follow-up ma- 
trix are to be 'itudled, the next step In the study procedure Is the 
determination of the number of students to be contacted whether 
all or only part of them. A discussion of the use of smnpllng In 
follow-up studies Is presented In Appendix A of this mantial, and 
Its reading Is recommended. Suffice It to say at this point that 
several factors must be weighed In arriving at a decision regard- 
ing sampling: (1) the overall scope of the study; (2) resources 
available; (3) the degree of generalization envisioned; (4) the 
number and variety of maiors to be Included. Before rejecting 
sampling techniques completely. It might be well to reflect on the 
fact that professional measurers of public opinion now can and do 
make statements with demonstrated precision about millions of 
people by obtaining data from only a few hundred. 

(6) Prepare Target List of Names 

Following the sampling decision, the study director now has two 
tasks to perform In the sequence leading to completion of the study: 
(1) select the medhod(s) to be used (see 6A below) ; and (2) pre- 
pare a target list of names of former students to be contacted. 
This list can (If desired) be prepared In the form of mailing labels, 
vhlch can be designed to Include codes Indicating major, sex and/or 
other desired characteristics. 
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-B) Select MethodCs), Select or Construct Instruments 

It is expected that In most studies at most colleges, mailed 
questionnaires will be the most feasible vehicle for obtaining 
data from former students • Other possibilities are telephone 
or in-*person interviews ^ or combinations of any of the three* 
Following are some suggestions ^ gleaned from the literature » 
for preparing instruments and conducting a follow-up study by 
mail: 

1. design the instrument to require the minimum possible 

time to complete 
2« use as fe^ words as possible in asking the questions 
3« use faiuiliar words, don't abbreviate, avoid long ex- 
pressions 

4« use two simple questions rather than one complex one 
5« avoid hints as to desirable or expected answers 
6. phrase questions to avoid academically or socially ac- 
ceptable responses 
7* place easiest to answer questions near the beginning 
8« sequence questions to conform with respondent's way of 
thinking 

9« plan method of tabulating results during questionnaire 
preparation 

10 « consider using double postcard for brief questionnaire 
11. observe parallelism and exclusiveness in multiple-choice 
questions 

12 « make directions brief but complete and set off with dis- 
tinctive lettering 

13 « state a deadline of about ten days after mailing 

14« don't ask a question unless it clearly relates to the 
purpose of the study 

15 • indicate endorsement of the questionnaire by someone held 
in high esteem by the recipient 

16 • try out yonr questionnaire on a few people before you 
finalize it 

17« send questionnaire so it will arrive early in the week 
18 • if a second mailing is performed, enclose another ques- 
tionnaire 

19 • send any second mailings about two weeks after first 
mailing 

20« enclose accompanying letter (except in double postcard mail- 
ing), briefly explaining the project and motivating the re- 
cipient to respond 

21« enclose a self-addressed postage paid envelope 

22 « give prime consideration in designing the questionnaire 
to motivating its return, secondary consideration to data 
processing requirements 

23 • if deemed necessary, assure respond€^nt that names will not 
be used in any published reports; however, most researchers 
agree that anonymity is not cril ^cal in response rates and 
it is not advisable to be able t make a second contact with 
non-respondents, thus requiring some kind of identification 

24 • use different color questionnaires for groups you want to 
study separately 
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Prepare Time Schedule 

After the target list of names Is prepared and methods and 
Instruments are decided upon, the study Is ready to "go." 
It Is suggested at this point that the study director pre- 
pare a rigid time schedule for the various steps In the study 
Implementation. Deadline dates can be set In accord with tha 
following time estimates. 



ACTIVITY £AY 

1. mall questionnaire 1 

2. mall second questionnaire (optional) 15 

3. telephone non-respondents (optional) 25-30 

4. collect responses, tally, and/or 

keypunch • • 3-30 

5. analyze data 30-35 

6. prepare and disseminate reports 36-40 



Thus tht study should be completed In a maximum of six weeks. 
Elimination of step 2 and/or 3 can reduce the time signifi- 
cantly but at some risk to generalizations from the study 
findings . 

Collect Data 

Among the ways in which In follow-up studies differ from other 
studies Is that the data arrive over a period of time. While 
having some obvious disadvantages, this spread of data collec- 
tion over a time period offers the advantage of utilizing a 
more leisurely tallying procedure. If carefully thought out 
formats are prepared In advance for recording the data as it 
arrives, secretarial help can usually keep tallies up to date 
during slack periods. For this reason, manual data processing 
may be preferable to electronic data for tallying and analyz- 
ing follow-up responses. A generally satisfactory method Is 
to use several blank questionnaires on which to mark tallies. 
Each blank questionnaire could be marked to identify desired 
sub-groupings. For example, if sub-groups are desired for sex 
(male and female), status (full-time, part-time) and age (un- 
der 21, 21-25, over 25) twelve (2x2x3) blank questionnaires 
should be used. For additional distinctions, different colored 
pencils can be used. For a larger number of breakdowns, punched 
cards and electronic data processing equipment are probably ad- 
visable. 

Analyze Data 

After data collection (including tallying) is completed, the 
study director (or someone else) must take a good hard look at 
the data with a view towards answering these questions: 

1, What concepts that we hold about our program(s) do the data 
confirm or deny? 
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(9) Analyze Data (continued) 

2, What do the data tell us about our program(8) that we 
didn't already know? 

3, What should be done as a result of the answers to ques- 
tions (1) and (2)? 

There are several possible approaches to analyzing the data. 
Again, the specific purposes of the study, Its scope, limi- 
tations on time and resources are factors which must be con- 
sidered. A simple tally may suffice one time, while a sophis- 
ticated statistical analysis may be appropriate another time. 
A thorough analysis In one office may be best at one time, 
while sending relevant questionnaires* to specific program 
heads for their analysis may be more appropriate at another. 

It is suggested that highly sophisticated statistical analy- 
ses are seldom appropriate, and It is not essential that the 
study director be able to perform such analyses; however, com- 
petent statisticians can nearly always be found on a college 
staff and their consultation Is advised before drawing In- 
ferences from "close" data that might have major Impact upon 
a program. 

(10) Disseminate Findings 

As in other phases of follow-up, there are no hard and fast 
rules about dissemination of findings — other than data In 
a deskdrawer or reports on a shelf are not very useful In 
evaluating or planning programs. The key point Is that per- 
sons in position to make or Influence program modifications 
must know the results of the study. For some, formal reports 
are preferred; for others, brief memoranda; for still others a 
simple presentation of the data In tabular or graphical form 
will best present the message. 

(11) Make Appropriate Program Adjustments 

This final step In the follow-up flowchart Is the culmination 
of the efforts -- this Is what it's all about. In one sense, 
this step is not part of the study, it's what should be done 
as a result of the study. In another sense. It Is the study. 

II-C Data for Vocational Education Follow-up Form 

This section presents reconmended procedures for obtaining specific 
Information required for completion of the Vocational Education Follcw- 
up Form (CCOE45) . The form requests numbers of students who have com- 
pleted their programs and have left college, numbers of students who 
have completed occupational programs but have left with marketable 
skills, and certain employment Information about the progrtto com- 
pletors. 

*It is almost always appropriate to send copies of written conraents to pro- 
gram heads 
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Data for Vocational Education Follow-up Fonn (continued) 

.Using the concept that "completion means fulfillment of program 
requirements that enable a student to enter on a competitive level 
the occupation for which he has been trained,"* this model defines 
Class 1, 2, and 3 Graduates, Class 1, 2, and 3 Certificates, and Class 
1 Achievers as program completers. Class 2 Achievers and Class 1 Par- 
tial Completers are considered to have left their programs with market- 
able skills. 

The accompanying minimum questionnaire provides a vehicle for obtain- 
ing the required employment Information from completors, and the accom- 
panying flowchart outlines a set of procedures leading to completion of 
the form* 



* p. 10, Instructions for Preparing Occupational Education Statisti- 
cal Reports for 1972-73, Chancellor's Office, California Conrounlty 
Colleges. 
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OCCUPATIONAL STUDENT FOLLOW-UP QUESTIONNAIRE 
(Mlnlint&n questions for CCOE45) 



1, What is your present employment status? 

working, full-time (30 hours per week or more) 

(1) 

working, part-time (less than 30 hours per week) 



(2) 

not working, looking for a job 



(3) 

not working, not looking for a job 



(4) 

military service 



(5) 

2, Which single statement best describes your present job? 

_____ In the occupation for which I prepared while In college. 
In an occupation related to my college training. 



(2) 

In a field not related to ray college training. 



(3) 

apprpnticcshlp program (specify) 



(4) 

not employed « 



(5) 

3« Are you attending college? 

no 

(1) 

yes 



(2) College 

Major 



No* of units carried 



BEST COPY AVMIABIE 
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FLOWCHART FOR COMPLETING CC0E45 



Prepare 
Follow-up 
Matrix 



select 
G1,G2,G3 
Cl,C2,C3 ,A1 



I 



Prepare 
Mailing 
Labels 



Select: 
A2, PI 



^ 

Send 
Questionnaires 



Tally 
Responses 



Tes 
for 
Bla 


t 

ie 
S 




No 

r 



to * 
Total 



Yes 



RtR8 

Additional 
Contacts 



Mako 

Counts 
Cols. 1-12 



Make 
Counts 
Col. 13 



Complete 
CCOE45 



} 



o 
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* See Appendix A, pp. 5-7 for suggested methods 
Repeat for each semester (quarter) and totals 



Page 30 « 



Notes on Flowchart for Completing CCOE45 

(1) Groups to be selected from Follow-up Matrix are those de- 
fined as Program Completors (Gl, G2, Cl, C2, C3, Al) and 
those with marketable skills (A2y Pl)« The latter requires 
counts only* 

(2) The form requires figures for all occupational majors ac- 
cording to program* Sampling Is usually not appropriate 
for this purpose* However, it Is possible that much of 
the Information can be obtained from students during the 
last few days of the semester* Thus, 1007* mailing (or 
telephoning If desired) may not be required* 

(3) It Is recommended that mailing labels Indicating codes for 
sex and major be prepared, and that the questionnaire be 
designed so that the mailing label will be returned attached 
to the completed questionnaire* 

(4) Two to three weeks should be allowed for questionnaires to 
be returned* At this point some check should be made as to 
*1>las" of responses* Some suggestions for testing response 
bias are given In the appendix, but a general look at dis- 
tributions of sex, major, mailing areas will often be suf- 
ficient to conclude that the representativeness of the re- 
sponse Is adequate* If bias clearly shows » It may be ad- 
visable to mall second questionnaires to and/or telephone 
non-respondents* Although subtle biases are difficult to 
detect, significant biases are usually not and responses 
should be at least cursorily examined before response fig- 
ures are expanded to the total mailing* 

(5) When assured that response bias Is suitably low, percentage 
figures for the responses should be calculated and the pro- 
ducts of those percents multiplied by the total numbers In 
the respective groups should be the ntmbers reported on the 
form* 
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II-D Data for Program Evaluation and Planning 

In accord with the guidelines stated at the outset, follow-up 
approaches are suggested In this section for five clusters In 
the follow-up matrix as follows: 

1. Graduates, Class 1, 2, 3; Certificates, Class 1, 
2, 3; and Achievers, Class 1 

2. Achievers, Class 2; Partial Completors, Class 1 

3. Achievers, Class 3 

4. Partial Completors, Class 2, 3 

5. Non-Coopletors , Class 1, 2, 3 

1. Graduates, Class 1, 2, 3; Certificates, Class 1, 2, 3; and 
Achievers, Class 1 

Theb>i students were Identified in an earlier section as 
"program completors" (for CCOE45). 

In order to determine types of Information about these stu- 
dents useful to occupational administrators, members of the 
consortium were requested to Indicate the five most Impor- 
tant (In order) questions they would like to ask occupational 
program completors. Questions were then resubmitted to the 
members, together with Information as to their ranking, with 
a request that they be rated on a 0-100 Inqiortance scale. 
Following Is an analysis of the results of that procedure. 

"Consensus" Questionnaire for Occupational Graduates, Certificates, and Achievers 

Questions were suggested by consortltmi meinbers and then rated on an Im- 
portance scale of 0 to 100. Figures given are median and "adjusted" 
range (extremes omitted) for the nine who responded. A high (low) me- 
dian score Indicates a high (low) level of Importance. A large range 
Indicates wide divergence of opinion, a small range Indicates consensus. 

Three questions are labeled "Essential," as they are required to produce 
data for the CCOE45 form. All had median scores of 100. Three athers 
are labeled "Highly Desirable," each with median score o* 95 of above. 
Four are labeled "Desirable" with median scores In the 8( 's. The re- 
mainder are labeled "Optional." 
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Ensentlal Questions 
(median, range) 

1. (100,0) What Is your present employment status? 

Working, full-time (30 hours per week or more) 

Working part-time (less than 30 hours per week) 

Not working, looking for a job 

Not working, not looking for a job 

Military Service 

2. (100,10) Which single statement best describes your present job? 

In the occupation for which I prepared while In college 

, In an occupation related to my college training 

In a field not related to ray college training 

Apprenticeship progreun (specify) 

Not employed 

3. (100,10) Are you attending college? 

No 

Yes 

Col lege 

Major 

Units carried 



Additional questions for those employed 
Highly Desirable: 

4, (100,20) How well did your college occupational training pre- 

pare you for skills you need on your present job? 

poorly fairly well very well 

5, (96,20) Do you feel that your college occupational training 

was lnq)ortant to you In getting your present job? 

^not at all some a great deal 

6, (95,30) What Is your current rate of pay*? 

$1.99 or less per hour 

^$2,00 - 2,99 per hour 

$3,00 - 3,99 per hour 

$4,00 - 4,99 per hour 

$5,00 or more per hour 

Desirable: 

7, (88,50) Please name the three college courses that were of 

most help to you In your work. 



* monthly or weekly units may be preferable If part-time and full-time 
workers are kept separate. 
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Desirable Questions (continued) 

8» (86 #80) Please rate the following according to your college 

experience. 

Excellent Average Poor 

CTass Instruction •••• 

Help from Instructor 
outside of class.*** 

Program planning as- 
sistance • •••••••••• 

Vocational Information 
available.. • • 

General opinion of 
college 



9. (80,27) Which one of the following was most helpful In 

assisting you In obtaining your first Job after 
leaving college? 

Parents I friends , reljatlves 

The college 

Employment agency (private) 

Employment agency (public) 
Other (specify) 
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10. (80,35) Were you employed prior to taking your work at 

college? 

Yes No 

At a lower level? Yes No 



In the same level? Yes No 



Optional: 

11. (75,30) How long did It take you to find your fltst Job 

after leaving college? 

I had It before I left college 

A month or less 

One to three months 

____ More than three months 

12. (75,60) Do you plan to continue your college education to 

a Bachelor's degree or higher? 

Already have Bachelor's degree 

Yes, will continue right away 

Yes, but later 

Probably not 

Definitely no 



13* (70,60) Please name the three college courses (of two 

units or more) that were of least help In pre- 
paring for your Job* 
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Optional Questions (continued) 

14, (70,60) If not employed in a field related to your oc- 

cupational training, was your occupational 
training of value to you? 

Yes No 
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If yes, in what way? 

15. (60,30) Do you think you will return to this or any 

college for additional training in order to 
advance on the Job? 

Yes No Not sure 



16. (60,35) How many jobs have you held since leaving college? 

This one is my first 

Two Three or more 



17. (55,25) What was the most important reason you stopped 

attending college? 

18. (50,50) How long do you expect to remain on your present 

job? 

Less than one year 

1-2 years 

_____ 2-5 years 
More than 5 years 



19. (50,80) Would you be willing to complete a more detailed 

questionnaire? 



Yes No 



20, (50,80) If not employed, would you like help in getting 

a job? 

Yes No 

(Name and address of employer is considered 
optional as several respondents requested its 
omission) 

It is suggested that the minimum questionnaire discussed in the pre- 
ceding section be expanded to include the questions rated as "highly 
desirable" and others selected by the college. 

It is also suggested that, where desirable and feasible, graduates 
and certificates *ino are continuing at the college (presumably for 
non- occupational courses) be included in this group. 



Page 35. 



"Conaenstts" Questionnaire (continued) 

Also, any In-depth questionnaire should provide one or more "open- 
end" questions requesting comments regarding strength and weaknesses 
of the particular program and/or the college In general. 

In line with the ansuraptlons stated at the outset of this . manual, it 
Is appropriate that a large share of the college resources available 
for follow-up studies be directed toward the cluster just described. 
It should by no means be Inferred that other groups In the follow-up 
matrix should receive no attention. To the contrary. It Is essential 
In an overall follow-up plan that Information be obtained from partial 
completors and non-completors . 

Following are some suggestions for obtaining Information about other 
clusters Indicated at the beginning of this section. 

2. Achievers, Class 2, Partial Completors, Class 1 

This group was defined (for the Voc-Ed form) to be those 
leaving an occupational program with marketable skills, de- 
pending upon the numbers Involved, resources available and 
pertinence at the time, this group could be sent questlon- 
narles Imllar to those used above. ' 

Questions relating to employment status, relevance of col- 
lege preparation, and opinions as to strengths and weaknes- 
ses of the college program could produce useful Information. 
However, these students left before completion of an occupa- 
tional program and questions relating to reasons for this pre- 
mature (apparently) leaving should surely be Included. 

Although the two groups (Class 2 achievers and Class 1 par- 
tial completors) are not distinguished for reporting on the 
CCOB45 form. It may well be advisable to separate them for 
follow-up planning purposes. 

3. Achievers, Class 3 

These non-contlniTlng students completed one (or more) Priority 
D occupational courses. It is suggested that these students be 
contacted, possibly on a sampling basis, to determine reasons 
for non-attendance, opinions regarding college work, current ac- 
tivities and their relation to college vork. It Is pos^jUle that 
many of these students had no intention of pursuing a two year 
(or less) occupational major. 

4. Partial Completors, Class 2, 3 

In this group are students who were classified as occupatlonjl 
majors but who left without completing any occupational courses 
(although completing some work). It Is suggested that th's 
group be contacted periodically, on a sampling basils, with ques- 
cions designed to elicit reasons for non-completion of occupa- 
tional courses. 



"Consensus" Questionnaire (continued^ 
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5. Non-Completors, Class 1, 2, 3 

This group could prpbably justifiably be labeled "drop- 
outs," as they completed no work. They will probably be 
the most difficult to contact, but should not be Ignored 
In the college plan for follow-up. 

It Is suggested that sampling Is appropriate for this group, 
with occasional In depth interviews considered as a possible 
method for probing Into reasons for non-completion and leads 
for possible Improvement of the college program. 

It should again be emphasized that the above clustering may 
not be the most appropriate one for a particular college at 
a particular time. It Is strongly reconnended that each col- 
lege set up procedures for routinely producing the follow-up 
matrix and then designing a follow-up program dependent upon 
the numbers in the various cells In the matrix, as well as up- 
on particular needs and resources of the college. 



Epilogue 



This manual has been addressed to the problem of 
short-range follow-up of community college occupational 
students and has considered only the students as sources 
of Information^ Inferences as to the value of either 
long-range follow-up or Information from sources other 
than students should not be made as a result of these 
omissions. The omissions were made simply because short- 
range follow-up with students as Information sources Is 
clearly of Immense value and, for whatever reason, has 
not been carefully done on a systematic basis at most col- 
leges • 



The systGsn developed In the manual for Identifying 
and classifying occupational students obviously cannot 
produce such Information with 1007« accuracy* It Is the 
collective judgment of project consortltm menbers, how- 
ever, that errors will be minimal as conqpared to most other 
systems now In use at various colleges. This judgment 
needs to be tested, and users of S.A.M. are urged to make 
such tests as appropriate to their situation and xeport 
findings to any member of the project consortium* 



It Is the hope of the people Involved In developing 
the model that It will be of assistance to the colleges, and 
not a burden « Comnents and suggestions toward Its Improve- 
ment will be welcomed « 
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APPENDIX 
SAMPLING IN FOLLOW-UP STUDIES 

Two of the knottiest problems In follow-up studies are: (1) guarantee- 
ing that those responding In the study are representative of the population 
under study, and (2) cost (In terms of money, personnel and time) of making 
contacts with former students and obtaining appropriate Information from them* 

A carefully structured sampling plan can often help In solving both of these 
problems. The following pages offer some suggestions for use of sampling In 
conducting follow-up studies. 

Some Important points to note: 

(1) For any sample to be representative of a popu- 
lation within desired probability limits, th.^, 
sample must be selected randomly , l.e^, every 
member of the population being sampled must 
have the same chance of being selected In the 
sample • 

(2) Sample size Is much more Important than what 
percent It Is of the population. A ten percent 
sample of ten Is meaningless; a ten percent sam- 
ple of 100,000 Is wasteful. Note (from Table 

p. 3), for example, that a random sample of 600 
gives results within + 4%, even if the popula- 
tion were Infinitely Targe. But also note, that 
for a small population of 100, the same accuracy 
requires a sample of 86. 

(3) Simple random sampling(see p. 4) is probably the 
most appropriate method for most follow-up studies. 

(4) Proportional stratified random sampling (see p. 8) 
is useful in certain studies where a large hetero- 
geneous group can be easily partloned into sub- 
groups, each of which is Internally homogeneous 
(relative to the total) . 

Some advantages of stratified sampling are: 

(a) provides data of known precision for sub* 
populations 

(b) often more convenient to administer 

(c) may bring about gain in precision of esti- 
mates for the whole population 

Stratified sampling can occasionally be useful in 
broad scale follow-up studies, but its usefulness 
is severely limited when many differing majors are 
to be included, as Is usually the case In occupa- 
tional follow-up. 



* precision will be less, however, as lamples are smaller 
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(5) Tables on the following pages apply to data In- 
volving attributes (not measurements) of people 
Involved (e.g., percent male, percent working, 
percent working In field, percent giving favor- 
able response). If data Involves measurement 
and produces averages (e.g., average monthly sal- 
ary, average number of hours per week employed, 
average on a rating scale) , sample sizes can be 
even lower than those Indicated. 

The Items listed below are presented on the following pages: 

Table for Selecting Sample Size 

Procedure for Selecting a Random 
Sample 

Procedure for Identifying Follow- 
up Response Bias 

Table for "Check" Questions 

Procedure for Extrapolating Sampl- 
ing Results to Entire Population 

Procedure for Selecting a Propor- 
tional Stratified Random Sample 
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TABLE FOR SELECTING SAMPLE SIZE 



If your popula* 
tlon Is of size 



and you want to guarantee* that 
your sample percent reflects your 
population percent within- 





+17. 


+2% 


+37. 


+47. 


+57. 


+67. 


+77. 


+87. 


+97. 


+107. 


100 


99 


97 


92 


86 


80 


73 


67 


61 


55 


/ A 

49 


150 


148 


142 


132 


121 


108 


97 


85 


76 


67 


59 


200 


196 


185 


169 


151 


132 


115 


99 


86 


75 


65 


250 


A/ / 

244 


A A ^ 

227 


AAA 

203 


4 

177 


4 P A 

152 


4 A /\ 

130 


4 4a 

110 


94 


81 


^A 

70 


AAA 
300 


A A 1 

291 


A^ ^ 

267 


A A C 

235 


A A4 

201 


169 


4 / A 

142 


4 4 A 

119 


101 


Q C 

03 


^A 

73 


A P A 

350 


A A O 

338 


306 


A^ /. 

264 


AAA 

222 


184 


1 CA 

152 


1 A ^ 

126 


1 A^ 

106 


OA 

89 


76 


400 


A A P 

385 


A / A 

343 


A A 4 

291 


A / 4 

241 


4 A X 

196 


4^4 

161 


4 AA 

132 


4 4a 

110 


A A 

92 


78 


450 


430 


379 


317 


A P A 

258 


A /\ A 

208 


4 X n 

168 


4 A ^ 

137 


4 4 A 

113 


A/ 

94 


OA 

80 


500 


476 


414 


A# 4 

341 


A 

273 


A 4 A 

218 


4 

174 


4/4 

141 


4 4^ 

116 


A ^ 

96 


A 4 

81 


600 


565 


/ A 4 

481 


A n p 

385 


301 


AA P 

235 


4 n p 

185 


4 / A 

148 


4 A 4 

121 


100 


OA 

83 


700 


653 


e / A 

542 


423 


A A / 

324 


A/ A 

249 


194 


4 P / 

154 


4 A / • 

124 


4 AA 

102 


O P 

85 


800 


739 


601 


458 


A / A 

343 


A ^ A 

260 


A /\ 4 

201 


4 m 

158 


4 A«> 

127 


4 A/ 

104 


o^ 

86 


900 


823 


655 


489 


M ^ 4 

361 


270 


206 


4^4 

IGl 


4 A A 

129 


4 AP 

105 


o^ 

87 


1000 


906 


706 


517 


376 


A 4 A 

278 


A 4 4 

211 


4 ^ / 

164 


4 A 4 

131 


106 


O A 

88 


4 AAA 

1200 


1067 


801 


565 


# a4 

401 


AAA 

292 


A 4 A 

219 


4 ^ A 

169 


4 A A 

134 


4 AA 

108 


OA 

89 


1400 


1222 


885 


606 


# A 4 

421 


A A A 

302 


A A P 

225 


4 4A 

172 


4 A^ 

136 


4 4 A 

110 


A A 

90 


1600 


1372 


961 


641 


437 


310 


AAA 

229 


4 

175 


4 A n 

138 


4 4 4 
111 


A 4 

91 


1800 


1516 


1029 


670 


451 


317 


233 


4 

177 


4 A A 

139 


4 4 A 

112 


AA 

92 


2000 


1656 


1092 


696 


/ X A 

462 


323 


A A X 

236 


4 ^A 

179 


4 / A 

140 


4 4 A 

112 


AA 

92 


2500 


1984 


1225 


A 

748 


/ A P 

485 


333 


A / A 

242 


4 AA 

182 


1 / A 

142 


4 4/ 

114 


A A 

93 


3000 


2286 


« A A # 

1334 


788 


P A 4 

501 


a/ 4 

341 


A/ P 

245 


4a/ 

184 


4 # A 

143 


4 4 P 

115 


94 


3500 


2566 


1425 


818 


513 


347 


248 


186 


144 


115 


94 


4000 


2824 


1501 


843 


522 


351 


251 


187 


145 


116 


94 


4500 


3065 


1566 


863 


530 


354 


252 


188 


146 


116 


95 


5000 


3289 


1623 


880 


536 


357 


254 


189 


146 


116 


95 


6000 


3693 


1715 


906 


546 


362 


256 


190 


147 


117 


95 


7000 


4049 


1788 


926 


553 


365 


257 


191 


147 


117 


95 


8000 


4365 


1847 


942 


559 


367 


259 


192 


148 


117 


95 


9000 


4647 


1896 


954 


563 


369 


260 


192 


148 


118 


96 


10000 


4900 


1937 


965 


567 


370 


260 


193 


148 


112 


96 


15000 


5856 


2070 


997 


578 


375 


263 


194 


149 


118 


96 


20000 


6489 


2144 


1014 


583 


377 


264 


195 


149 


118 


96 


25000 


6939 


2191 


1024 


587 


379 


264 


195 


150 


119 


96 



then you 
should use 
a sample of size 



Infinitely 

large 9604 2401 1067 600 384 267 196 150 119 96 

For example, If you have a population of 5,000 
people r .id you want the sample percent to be 
within 47. of the true population value, you 
should use a sample size 536. 

* odds 19 to 1 
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SUGGESTED PROCEDURE FOR SELECTING A RANDOM SAMPLE 



!• Determine the sainple size desired (see Table)* 

2* Assign a number to each member of the population* 

3* Divide the population size by the sample size and 
round down to nearest whole nxnnber, obtaining the 
"skip*' nxiraber* 

4. Select a "start" ntunber (smaller than or equal to 
the "skip" nimiber) by random procedure (e*g*, take 
the next to the last digit of your telephone number)* 

5* Draw the sample by selecting the person assigned the 
start number and succeeding persons assigned numbers 
obtained by successively adding the "skip" number* 

Example: 

It Is decided that sampling Is appropriate 
for a population of 3,500 and accuracy Is 
desired within + 4%* From the table, the 
sample size Is determined to be 513* Skip 
number Is 3500 -~513 • 6* Take start num- 
ber to be 5* Sample consists of those with 
numbers 5, 11, 17, 23, 29, 35, **** (every 
6th until 513 obtained)* 
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SUGGESTED PROCEDURE FOR IDENTIFYING FOLLOW-UP RESPONSE BIAS 

(Mail or Telephone) 



1. Include two or three "check" questions* for which 
total population percentages are known, e.g. , sex, 
ethnic, high school source, etc. 

2. Compare response percents with population per cents 
on "check" questions. 

3. If "check" question differences are all within al- 
lowable limits (see Table) , assume 7:espon8e sample 
is representative. 

4. If one or more "check" question differences are out- 
side allowable limits, response sample cannot be as- 
sumed representative and you should do one of th& 
following: 



(a) make further efforts at increasing 
number of responses 

(b) abandon the project 

(c) prepare report, clearly indicating 
the nature of the bias and indicat- 
ing that findings can not be general* 
ized 



*It Is of coxnrse best to use as "check" questions charac- 
teristics which are related to those which are of concern 
In the follow-up study. The "check" procedures indicated 
do not guarantee lack of bias, but should aid in reducing 
It. 
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TABI£ FOR "CHECK" QUESTIONS 



If you mailed to (or 
phoned) this many 




* odds are 19 to 1 

Example: 

If total mailing Is known to he 567. 
male, and you sent 250 questionnaires, 
a 607. response can he assumed to he un- 
hlased (at least as far as sex Is con- 
cerned) If the respondent percent male 
shows hetween 517. and 617. male (567.±57.). 
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PROCEDURE FOR EXPANDING FROM RESPONSE SAMPLE TO POPULATION 



!• Compare population and response sample Values on 
''check** qtiestlons* If any differences are outside 
allowable limits » extrapolation should not be made« 

2« If response rate Is less than 80% , '^oversampllng*'^ 
Is reconoended, e«g«f additional people should be 
contacted. To determine the number of additional 
contacts, divide the original sample size by the 
response rate (to get total necessary) and sub- 
tract the nisnber of responses already obtained. 

Example: 

If a sample of 600 produced a 75% response, 

/ 

a total of 1^ - 800 (or an additional 200) 

people should be contacted • 

3« If **check" questions indicate lack of bias and num- 
ber of responses is increased by 'Wersampling** (if 
necessary) percents and averages calculated in sample 
can be considered reliable estimates of the true popu- 
lation val\ies<» 



* This obviously can not be done if initial contacts were 
made to the entire population* Also, for low response 
rates, the recommended oversampling can be rather large* 
Compromises based on judgment and experience will have 
to be made* 



PROCEDURES FOR SEIilCTING A PROPORTIONAL STRATIFIED RANDOM SAMPLE 



Decide on the stratification variables, 

e.g., 4 levels of major, received (or not) Associate Degree 

Partition the population Into appropriate strata^ 

e«o«9 population of 5,000 stratified on degree and major might be: 



Degree ? 


Occup . 
CI. 1 


Occup . 
CI. 2 


Occup . 
CI. 3 


Non- 
Occup. 


Totals 


Yes 


600 


800 


300 


1300 


3000 


No 


600 


500 


200 


700 


2000 


Totals 


1200 


1300 


500 


2000 


5000 



Determine the proportion of the population In each stratum by divid- 
ing the number In the stratum by the total number in the population, 
e.g., 



Degree ? 


Occup . 
CI. 1 


Occup . 
CI. 2 


Occup . 
CI. 3 


Non- 
Occup. 


Totals 


Yes 


12 


16 


6 


26 


60 


No 


12 


10 


4 


14 


40 


Totals 


?4 


26 


10 


40 


100 



(figures now Indicate percents) 



Determine total sample size desired (from table or other considera- 
tions) e.g., assume In this example sample of 400 ueslred« 

Determine stratum sample sizes by multiplying total sample size by 
percents obtained In step 3. 



Degree ? 


Occup. 
CI, 1 


Occup . 
CI. 2 


Occup . 
CI. 3 


Non- 
Occt.p. 


Totals 


Yes 


48 


64 


24 


104 


240 


No 


48 


40 


16 


56 


100 


Totals 


96 


104 


40 


160 


400 



Select sample for each stratum, using random sampling procedures* 



(Sample In-Class Questionnaire) 



To the student: 

The purpose of this questionnaire Is three-fold: (1) to provide 
necessary Information for funding purposes; (2) to provide the 
college Information to help us better meet student needs, and (3) 
to be of assistance to you In career planning* If you have com- 
pleted this form In another class , It Is not necessary to complete 
another one# 

When you later receive a follow-up questionnaire, ve would appreci- 
ate your completing and returning It pron^tly* Thank you* 



STUDENT PLANS FOR NBCT SEMESTER 

Instructo r 

Course 

Date 

Name College Malo r 

PERMANENT address 

(Street) (City) (State) ZIP 

Telephone Nimiber 

Indicate below relative (or friend) who will always know your address* 



(Name) (S treet) (City) (S tate) ZIP 

Please check (and complete) any of the following that are appropriate* 
(To the best of your knowledge at this time) 

I plan to return to this college next semester* 

I plan to transfer to (college) * 
I plan to work full-time next semester* 

I am fairly certain I will be employed 

with (name and address of employer) 



I think I may have a Job but am not 
sure* (Name and address of prospective 
employer) 



I have no prospects for employment at 
this time and I would appreciate assis- 
tance* 



(Satnple Exit Interview Form) 
Student Wtthdraval Form 
Name 



Date 

ID # 



Permanent Address 



College Major 



(Address) 



(City) 



ZIP 



Have you been employed this semester? 
1/4 time 1/2 time 3/4 time 



No 



Yes 



Do you plan to return to this college? Yes^ 



full-time 
No 



Do you plan to transfer to a four-year college? 

Yes No Not sure 



How well do you feel this college served you? 

excellently adequately^ 



poorly_ 



Please check (one or more) reasons for your withdrawal: 
insufficient funds 

personal or family Illness or accident 

transportation problem 



going to work full-time 
(h«vc you found a job yet? 



Yes 



No 



unhappy with schedule 
family movfng away 
going to another school 
conflicting jot* \ours 
not enough time i:o study 
personal I prefer not to state 
other (please describe) 



Net sure 



Please write ^ny comments on the back of this sheet* 



(Sample Follow-up Alert Letter) 



Dear : 

Our records Indicate that you are no longer In 
attendance at this college* 

We hope your stay has been worthwhile , and hope 
you will return whenever our offerings can be of 
service to you* 

One of our most important sources of Information 
to assist us In planning programs which can be of 
service Is feedback from former students like your- 
self* From time to time we send questionnaires to 
our former students to obtain this type of Informa- 
tion, and we hope that when you receive one of our 
requests you will respond promptly* 

Meanwhile, we should like to ask tliat you keep us 
posted on any change of address* 

We wish you well In your future endeavors* 

Sincerely, 



(possibly enclose Change of Address form) 
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